A Learning Agent for Parameter Estimation in Speeded Tests

The assessment of a person's traits such as ability is a fundamental problem in human sciences. Compared to traditional paper and pencil tests, computer based assessment not only facilitates data acquisition and processing, but also allows for real-time adaptivity and personalization. By adaptively selecting tasks for each test subject, competency levels can be assessed with fewer items. We focus on assessments of traits that can be measured by determining the shortest time limit allowing a testee to solve simple repetitive tasks (speed tests). Existing approaches for adjusting the time limit are either intrinsically non-adaptive or lack theoretical foundation. By contrast, we propose a mathematically sound framework in which latent competency skills are represented by belief distributions on compact intervals. The algorithm iteratively computes a new di culty setting, such that the amount of belief that can be updated after feedback has been received is maximized. We rigorously prove a bound on the algorithms' step size paving the way for convergence analysis. Empirical simulations show that our method performs equally well or better than state of the art baselines in a near-realistic scenario simulating testee behaviour under di erent assumptions.